DOCUMENT RESUME 



ED 426 649 



HE 031 752 



AUTHOR 

TITLE 

PUB DATE 

NOTE 

PUB TYPE 

EDRS PRICE 

DESCRIPTORS 



Knox, Alan B. 

Recognizing Excellent Outreach Performance. 

1998-12-00 
52p . 

Reports - Research (143) 

MF01/PC03 Plus Postage. 

Academic Rank (Professional) ; *College Faculty; Data 
Analysis; Evaluation Methods; * Faculty Evaluation; *Faculty 
Promotion; Higher Education; *Outreach Programs; Professors; 
♦School Community Relationship 



ABSTRACT 



This study analyzed aspects of assistant professor promotion 
and tenure packets that contained varying emphases on outreach associated 
with a positive promotion recommendation. Fictionalized versions of actual 
promotion packets for 18 men and women were obtained from seven midwest 
universities. Packets varied by type of department and by extent and type of 
outreach responsibilities reported by the assistant professors. Forty- three 
experienced reviewers evaluated packets, rating 20 aspects and listing the 
aspects most influential in their recommendation. Data analysis included 
inductive classification of written comments, and cluster analysis of the 
ratings of the 20 aspects and the recommendations. The study found that the 
aspects most generally associated with promotion were outreach research and 
external funding. The aspects most associated with assessing outreach 
performance were outreach research, outreach teaching, outreach service, 
letter of appointment, and impact on the field. Classification of open-ended 
comments regarding the promotion recommendation found the following most 
related: quality, research, and outreach impact. Implications for 
strengthening the process of outreach promotion are identified. The form for 
assessing the promotion packet is attached. (Contains 31 references.) (DB) 
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Overview 1 1/98 

RECOGNIZING EXCELLENT OUTREACH PERFORMANCE 

Alan B. Knox, University of Wisconsin 
1282H Edu. Sci., 1025 W. Johnson St., Madison, WI 53706 

The study purpose was to analyze aspects of assistant professor promotion and 
tenure packets that contained varying emphasis on outreach, which were associated with a 
positive promotion recommendation. This topic is important because outreach teaching and 
research is increasingly central to the university mission and excellent outreach programs 
reflect the quality of faculty outreach performance. 

Fictionalized versions of actual promotion packets for 18 men and women were 
obtained from 7 midwest Land Grant universities. They varied in type of department and the 
extent and type of outreach responsibilities that these assistant professors reported. The 
promotion packets included: appointment letter and position description, transmittal letter 
from the department, vita, sample publications, assessment of teaching performance, and 
external review letters. Faculty members and administrators with experience reviewing 
promotion materials read packets and completed an assessment form which covered 20 
aspects (12 sections of the packet and 8 overall impressions). They rated each aspect 
regarding importance to promotion generally and evidence of quality in that packet. They 
also made a promotion recommendation (deny, marginal, recommend, strongly recommend) 
then listed which of the aspects were most influential on their recommendation and provided 
comments on their rationale. Data analysis included inductive classification of written 
comments, and cluster analysis of the ratings of the 20 aspects and the recommendation. 

The cluster analysis yielded three clusters, the first of which contained two sub 
groups. The resulting four sets of aspects were: 

* Creatively responsive (7 aspects and the promotion recommendation) 

* Impact on the field (6 aspects) 

* Balance of research with teaching and service (18 aspects) 

* Role expectations, especially regarding teaching (7 aspects) 

Using the average level of promotion recommendation (based on ratings from 2 to 6 
reviewers) the distribution was 3 strong recommendations, 13 recommendations, and 2 
marginal recommendations. From a cross tabulation of the recommendation and aspects 
reported as influential, 6 aspects were associated with the recommendation regardless of 
outreach responsibility. They were: external review letters, resident teaching, transmittal 
letter, sample publications, and vita. The aspects that were most associated for promotion 
generally were: outreach research and external funding. The aspects that were most 
associated for assessing outreach performance were: outreach research, outreach teaching, 
outreach service, letter of appointment, and impact on the field. When the 20 aspects were 
compared with average level of recommendation (deny, recommend, strong), 3 aspects were 
especially associated when the emphasis was on outreach. They were external review letters, 
outreach research, and sample publications. Other aspects related to promotion generally 
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included: discipline research, resident teaching, transmittal letter, outreach teaching, and 
appointment letter. 

From the classification of open-ended comments regarding assessment decisions, 
those especially related to the promotion recommendation were: quality, research and 
outreach impact. Other comments pertained to: expectations, the organization of the 
promotion packet, and teaching. 

Although there were many similarities across reviewers and institutions, there were 
some contrasting viewpoints which reflected reviewers’ experience and values. Such 
differing viewpoints emerged especially regarding: 

* appointment/position description 

* transmittal of packet from department 

* external review 

Some reviewers emphasized an aspect as essential and other reviewers ignored it. 
These differing viewpoints suggest that one use of general guidelines is to focus attention on 
assessing actual expectations and past decisions in the specific situation. 

Increasing attention to assessing and recognizing the quality of faculty outreach 
performance reflects a growing interest in multiple faculty roles regarding knowledge 
discovery, integration, dissemination and application. 

Implications for strengthening the process of outreach promotion include: 

* Be responsive to reviewers’ expectations. 

* Use guidelines to help candidates, mentors, reviewers and administrators. 

* As the promotion packet evolves, emphasize descriptive and evaluative 
information about aspects relevant to the promotion decision (such as: 
creatively responsive, impact on field, balance of research and other 
responsibilities, and expectations related to teaching.) 

* Make the promotion packet brief, complete and clear. 

* Outreach forms of teaching, research and service are increasingly accepted 
for promotion. 

* High quality performance and accomplishment is more important than the 
proportion of outreach. 

* Contrasting reviewer viewpoints suggest treating guidelines as tentative 
pending assessment of actual current expectations and past decisions. 

* Consider likely institutional expectations about the importance of various 
aspects, as well as evidence of performance. 

* Universities can revise guidelines but reviewers’ views may not match 
guidelines. 

* Use writings about outreach promotion as background for guidance for 
candidates. 
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Purpose 

Judging the quality of faculty outreach performance has become 
increasingly important as extension, continuing education and other forms of 
outreach have become more widespread and central to the university mission. 

The century-long trend of increasing centrality of outreach in United States higher 
education institutions is reflected in various changes in policies and practices. One 
is that part-time and returning adult students have become the new majority in 
higher education credit and degree programs (outnumbering young full-time 
students entering directly from secondary school). A second policy change is the 
increasing number of university mission statements that include dissemination and 
application of knowledge along with discovery and synthesis as forms of 
scholarship. 

The excellence of a broad range of outreach forms of teaching, research 
and service reflects the quality of performance by faculty members who help to 
plan and conduct such activities. Full-time faculty members and adjunct 
instructors (who teach on a part-time or short-term basis) vary in their extent and 
types of contributions to outreach. Especially in Land Grant universities with 
almost a century of experience with extension and other outreach activities, some 
faculty and staff members are uninvolved and for some faculty and staff members 
their outreach contributions are a small proportion of their total responsibilities, 
but some faculty and staff members carry substantial outreach responsibilities. 
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The purpose of this study was to analyze the types of information in 
packets used to make promotion and tenure decisions for assistant professors with 
substantial outreach responsibilities. The object was to identify variables 
associated with positive promotion decisions. Such associations might be 
widespread or specific to institutions and scholarly fields. The study findings 
might have implications for institutional policies, career priorities, and 
interpersonal relations which could guide decisions by such assistant professors, by 
review committees at department, college and institutional levels, and by outreach 
administrators. 

Method 

The study methods and procedures were designed to approximate the 
process when promotion and tenure materials for faculty members with substantial 
outreach responsibilities are evaluated and promotion recommendations are made. 
The process began by contacting people known to care about outreach promotion 
at Midwest Land Grant universities. A written project overview was then sent to 
them and to people whom they had identified at their university who were in 
departments, colleges and campus-level positions that would allow them to 
suggest actual recent promotion packets that included outreach performance. 

As a result, dozens of faculty members who went through the promotion 
review process agreed to send a copy of their promotion materials for use in this 
study. These men and women were from one of seven universities, had 
appointments in more than a dozen departments, and varied greatly in the extent of 

O 
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their outreach responsibilities (ranging from less than 15 percent to full-time 
budgeted extension appointments). In addition to departments in colleges with a 
tradition of extension appointments (such as agriculture and home economics) 
packets were also obtained from departments in university professional schools 
and liberal arts colleges. Eighteen packets were fictionalized to retain the 
authentic character of the materials, but to obscure the name of the faculty member 
and of the institution, as was agreed upon from the outset. The fictionalized 
versions were sent to the faculty members, and each agreed that the final version 
was authentic and sufficiently fictionalized. (Most of them didn’t care whether the 
original or fictionalized version was used for the study, but use of a fictionalized 
version allowed reviewers to rate the packet as they might if it was from their own 
university). 

When fictionalizing each of the packets, a somewhat distinctive set of 
promotion materials was available, depending on practices in the institution and 
field. However, for ease of review, the format was somewhat standardized and 
some redundancy was reduced. Actual illustrative publications were included, but 
without explicit identification of the author (assistant professor). Because of 
assurances of confidentiality when external review letters were obtained, fictional 
versions were prepared with similar contents and recommendations, but with no 
reviewer identified. Typical contents of a promotion packet included: appointment 
letter or position description, a transmittal letter from the department chair with 
the departmental recommendation and vote, the faculty member’s vita, sample 
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publications, report on externally funded projects, descriptions and evaluations of 
performance in institutional and outreach forms of teaching and research and 
service, and external review letters. 

People at each of the Land Grant universities who helped to identify 
potential promotion packets also suggested potential reviewers. Such people with 
experience reviewing the promotion materials of assistant professors and making 
recommendations were contacted and were provided a listing of the promotion 
packets, including the assistant professor’s departmental field. Those who agreed 
were sent one or more packets, and an assessment form (questionnaire) to 
complete after reading the packet. Forty-three reviewers returned a total of sixty- 
two completed assessment forms. 

The assessment form was in three parts. Part A was a listing of twelve 
aspects of the promotion packet (such as the transmittal letter from the department 
chair, ratings of teaching, sample publications, external review letters). Part B was 
a listing of eight overall impressions based on information from various aspects of 
the promotion packet (such as creativity, methodological rigor, and contribution to 
the field related to outreach). For each of these twenty statements in Parts A and 
B, the reviewer was asked to check on a five point scale how important such 
information is for a promotion decision and the extent of quality as reflected in the 
information provided in the packet, and to provide explanatory comments. Part C 
contained the reviewer’s recommendation regarding promotion, identification of 
aspects of the packet that most influenced the recommendation, missing 
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information that should have been included, aspects most related to judging 
outreach; and other comments. Reviewers were assured of confidentiality of their 
reviews. 

Analysis of the assessment forms included the following procedures. Each 
completed assessment form was read for content and clarity, and where indicated 
the promotion packet to which it referred was consulted. The open-ended items in 
Part C were read and the following inductive classification was prepared and used 
to code the written comments in each assessment form. 

* Quality and productivity 

* Expectations for role 

* Vita and documentation 

* Research and publications 

* Teaching 

* Outreach 

* External review 

* Funding 

* Own assessment 

* None 

Cluster analysis was used to analyze the forty importance and quality 
ratings in parts A and B using a 5-point scale, and a 4-point scale for the 
promotion recommendation (deny, marginal support, recommend promotion, 
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strong recommendation). The results of this analysis were analyzed for individual 
promotion packets, for all eighteen, and for sub sets based on departmental field of 
the assistant professor and university of reviewer (where numbers of packets and 
reviewers per institution allowed). After relationships between the promotion 
recommendation and other variables were identified, illustrative examples from 
individual packets and assessment forms were selected. 

Findings 

Before reporting the findings related to the clusters of ratings that emerged, 
this section includes brief comments on. the promotion recommendation, the 
ratings on the importance of variables, and discrepancies between importance and 
quality ratings and the recommendation. 

The criterion variable in this study was the promotion recommendation. 
There were four levels of rating: 

1 . deny promotion 

2. marginal support 

3 . recommend promotion 

4. strong recommendation 

The number of reviewers who made recommendations for each of the 
promotion packets ranged from 2 to 6, with a mean rating of 3.2 for all 18 packets. 
The range of average ratings was from a high of 3.8 to a low of 2.0. The 
distribution of frequencies by quarters is listed in Table 1 : 
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Table 1 

Distribution of Promotion Recommendation Ratings 
Ave. Rating Frequency 

top quarter 



3.8 1 

3.7 1 

3.5 2 



middle two quarters 

3.3 6 

3.0 3 



bottom quarter 

2.8 3 

2.5 1 

2.0 1 

The ratings of the importance of a variable for promotion decisions 
generally were compared with the ratings for each variable regarding quality based 
on the information in the specific promotion packet. For most variables, the two 
average ratings were similar, with six or fewer of the packets in which the quality 
ratings were somewhat lower than the importance ratings. For three variables, 
there were more packets with discrepancies between average importance ratings 
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and the somewhat lower average quality ratings. They were institutional service 
(10 packets), outreach service (7 packets), and honors and recognition (7 packets). 

The average quality ratings of the total set of packets varied greatly. One 
way of indicating this variation was the percentage of ratings for each variable that 
was at the highest (5) level. Table 2 lists the 20 variables in descending order, 
including the percent with a 5 rating for the general importance of the variable, the 
percent with a 5 rating for quality, and the difference between the two ratings for 
that variable. 

Most of the ratings of the importance of each variable generally were 
somewhat higher than the ratings of the quality of the information related to that 
variable in a specific packet. The greatest discrepancies were for variables 6 
(expectations) and 4 (chair’s letter). For three of the lowest ranked variables 
regarding importance, the rating of quality was not lower than the rating of 
importance (20, institutional service; 17, external funding; and 19, honors and 
recognition). The four highest ranked variables regarding importance reflected 
performance and results. The nine high middle variables emphasized outreach 
along with expectations and performance that reflected a balance of teaching, 
research and service. The two lowest ranked variables (19, honors; 20, 
institutional service) were viewed by many reviewers as not to be expected for 
assistant professors. 
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Table 2 

Average Ratings of the 
Importance of Variables 



Percent of Raters 

Variable Assigning a 5 Rating 





Importance 


Quality 


Difference 


High 








1 . Impact and contribution to field 


67 


43 


24 


2. Responsive to outreach expectations 


64 


57 


7 


3. External review letters 


64 


43 


21 


4. Letter from department chair 


62 


32 


30 


High middle 








5. Balance, teaching, research, service 


54 


46 


8 


6. Expectations, appointment, position 


54 


13 


41 


7. Importance of topics 


52 


49 


3 


8. Discipline research, publications 


52 


38 


14 


9. Originality, creativity, innovation 


52 


33 


19 


10. Outreach research 


52 


29 


23 


1 1. Sample publications 


51 


35 


16 


12. Outreach teaching 


49 


25 


24 


13. Outreach service 


44 


38 


6 


14. Vita, career information 


43 


29 


14 


Low middle 








15. Methodological rigor 


35 


27 


8 


16. Resident teaching 


33 


19 


14 


17. External funding, projects 


30 


35 


(5) 


1 8. Career enhancement 


29 


21 


8 


Low 








19. Honors and recognition 


19 


19 


0 


20. Institutional service 


6 


19 


(13) 
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The central purpose of the study was to explore relationships between the 
variables (importance, quality) and the recommendation regarding promotion. The 
main way of doing so was cluster analysis. The quality ratings were emphasized 
when interpreting the clusters, but the importance ratings were also considered. 
Three major clusters emerged, with two sub-groups within each cluster (one of 
which was a single variable). The major cluster that included the promotion 
recommendation consisted of 14 variables. The sub-group of 8 variables within 
that cluster that included the recommendation reflected performance that was 
creatively responsive (based on the quality variables). The other sub-group of 6 
variables within that cluster reflected impact on the field. The second major cluster 
was of 20 variables, with sub-groups of 1 8 and 2 variables (the latter of which was 
composed of importance ratings for letter from department chair and outreach 
teaching). The cluster of 18 variables reflected career balance of research as well 
as teaching and service. The third cluster of 7 variables reflected role expectations, 
especially related to teaching. In addition to the cluster analysis for all ratings of 
all promotion packets, cluster analysis was performed for sub categories related to 
disciplinary fields of the assistant professors and to institutions of the reviewers. 
The following presentation of findings from the cluster analysis is based on the 
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total data set, but differences related to sub-categories are also noted. The 
variables grouped in each cluster and sub-group are listed in Table 3. 
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Table 3 

Variables Included in Four Clusters 



CLUSTER 


OUALITY 


IMPORTANCE 


RESPONSIVE 


Responsive to outreach 


Originality 




Originality 


Publication 




(included recommendation) 


External letters 
Honors 

Institutional service 


IMPACT 


Impact on field 


Impact on field 




Topic importance 


Topic importance 




External review letters 


Responsive to outreach 


CAREER BALANCE 


Vita, career information. 


Vita, career information 




Career enhancement 


Career enhancement 




Disciplinary research 


Disciplinary research 




Outreach research 


Outreach research 




Outreach service 


Outreach service 




Disciplinary rigor 


Disciplinary rigor 




Balance (teaching, research, service) 


Balance (teaching, research, service) 




Publications 

Honors 

Institutional service 


Funding of projects 
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EXPECTATIONS 


Expectations, appointment 


Expectations, appointment 


TEACHING 


Resident teaching 


Resident teaching 




Outreach teaching 






Letter, chair 






Funding of projects 





The creatively responsive subgroup of the first cluster included the 
recommendation regarding promotion for the total of all reviewers and all 
promotion packets, and also in most of the sub populations (and the 
recommendation was in an adjoining sub-group of the cluster for the remaining 
two). Three sub populations for which cluster analysis was conducted were based 
on the disciplinary fields of the assistant professors (7 from agriculture, 6 from 
home economics and 5 from professional schools and liberal arts). Although each 
of these sub populations had a similar creatively responsive cluster of quality 
ratings, there were some differences. For example, for agriculture faculty, this 
cluster included responsiveness to outreach but not creativity, and also included 
were outreach and institutional service, external funding, honors, outreach 
research, and topic importance. For home economics faculty, this cluster included 
creativity but not responsiveness, and also included topic importance and external 
review letters. For the faculty members from professional schools and liberal arts, 
the recommendation for promotion was not included in this cluster, but creativity, 
disciplinary research, and outreach service were. The pattern of variables in this 
and other clusters for faculty members from agriculture and home economics 
probably reflects the tradition of budgeted extension appointments in these fields. 
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Unlike reviewers from liberal arts and professional schools, many of whom 
seldom reviewed packets for faculty members with cooperative extension 
appointments, reviewers with such experience usually understood such outreach 
appointments. For the former, it is especially important that appointments and 
roles are clear, and that experience with outreach research indicates ability to 
supervise doctoral dissertations. 

Cluster analysis was also conducted for the sub populations based on the 
university in which reviewers were located, where there were 5 or more 
reviewers. There was less similarity across these sub populations than for 
disciplinary fields of the assistant professors. For example, quality ratings for 
responsiveness and creativity occurred in this cluster, but not the ratings of 
importance for these variables. This cluster for one institution included vita and 
honors, while another included external review letters and outreach service, and a 
third included outreach research. This pattern of institutional clusters probably 
reflects different institutional traditions regarding promotion and tenure criteria 
along with variation among disciplinary fields. 

The closely related sub group of variables within the first major cluster was 
labeled impact on the field. Three quality ratings constituted this cluster for the 
total set of all assistant professors and all reviewers. The aspects on the 
importance of the topics emphasized in outreach and other activities and the 
indications of impact on the field related to this specialty for both quality of 
performance and the importance of the variables were highly rated by the 
reviewers. The third quality rating in this cluster was external review letters which 
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many reviewers emphasized as major indicators of impact and prominence in the 
field beyond the institution. The importance rating of responsiveness to outreach 
(for which the quality rating was included in the other subgroup in this major 
cluster) was also included in this impact cluster. 

The clusters for the sub-populations were very similar. The quality ratings 
for agriculture faculty were impact on field and external letters, for home 
economics were impact and responsiveness, and for other faculty was topic 
importance. For one of the institutions, the promotion recommendation was in this 
cluster, along with impact, topic importance, external letters, and discipline- 
oriented research. For the other institutions with five or more reviewers, the 
variables usually included in this impact cluster were topic importance, external 
letters, responsiveness to outreach, and creativity. 

The second major cluster included half of the variables, so was quite 
varied, but the main theme was career balance regarding research as well as 
teaching and service, as reflected especially in the vita and publications. The 
promotion recommendation was not part of this cluster, with the exception of the 
sub population of assistant professors from professional schools and liberal arts. 
Other variables in this cluster with both quality and importance ratings included 
career enhancement, disciplinary research, methodological rigor, outreach 
research, and outreach service. The teaching related variables did not occur in this 
cluster. The pattern of variables was similar for each of the sub populations based 
on disciplinary field and institutions of reviewers. 
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The remaining cluster was similar for the sub populations as it was for the 
total. The focus was expectations , especially related to teaching . Quality and 
importance variables in this cluster included expectations from the position 
description, the letter of appointment, resident teaching and outreach teaching. 

The quality rating for external funding for projects was also included. 

In summary, four clusters or sub groups emerged. The promotion 
recommendation was associated with the creatively responsive sub group of the 
first cluster and the other sub group was impact on the field. Half of the variables 
were associated with the career balance cluster which emphasized research and 
service. The remaining cluster was expectations, especially related to teaching. 

The number of reviewers who made promotion recommendations for each 
assistant professor ranged from four packets for which there were two reviewers 
to one packet for which there were six. The categories of recommendation were: 
deny, marginal support, recommend, and strongly recommend. Fourteen of the 18 
packets received similar recommendations, either identical or adjoining ratings 
regarding strength of support for promotion and tenure. 
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Table 4 presents the numbers of packets rated, raters, and ratings in each of 



the four categories. 



Table 4 



Instituation 


Packets 


Number 


Deny 


Marginal 


Recommend 


Strong 


of Rater 


Rated 


of Raters 










A 


13 


13 


i 


i 


7 


4 


B 


12 


10 


- 


- 


10 


2 


C 


11 


7 


2 


- 


8 


1 


D 


11 


5 


- 


3 


2 


6 


E 


7 


3 


- 


- 


5 


2 


F 


6 


4 


1 


1 


2 


2 


G 


2 


1 


- 


- 


- 


2 


TOTAL 


62 


43 


4 


5 


34 


19 



Most of the ratings were in the recommend promotion category except one 
institution where a majority of the ratings were in the strong recommendation 
category. Raters from two of the institutions used all four categories, and raters 
from two institutions used three categories. The sole rater from one institution 
rated both packets in the strongly recommend category, and each of the ratings 
from raters at other two institutions were in the recommend or strongly 
recommend category. In general, the ratings were fairly consistent, but there were 
some variations by institution of rater. 
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To better understand the reviewers’ rationales for promotion 
recommendations, they were asked which of the 20 aspects of the candidates’ 
packet most influenced their recommendation and why. They had rated the 
aspects for importance and quality just before making their recommendation. 

In addition to asking about their rationale for influences on their 
recommendation generally, they were asked which aspects listed were most 
important to them in judging the performance of a faculty member with outreach 
responsibilities. There were, of course, many similarities. This section reports the 
aspects that were influential both generally and in relation to outreach 
performance, followed by the aspects that were mentioned more frequently as a 
general influence than in relation to outreach performance. The subsequent part 
reports aspects mentioned more frequently regarding outreach than generally. The 
section concludes with Table 8 which compares frequently mentioned aspects with 
level of recommendation. 
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Table 5 lists the aspects mentioned frequently both generally and regarding 
outreach performance. 



Table 5 

Influential Aspects 

Aspect Frequency of Mention 





General 


Outreach 


* External Review Letters 


30 


27 


* Resident Teaching/Peer Review/ 


22 


19 


Student Ratings 






* Transmittal Letter from Department 


21 


16 


Chair 






* Sample Publications 


19 


20 


* Vita/Background Information on 


16 


16 


Education, Career 







These aspects of a promotion packet and the performance as an assistant 
professor that it should reflect, might be considered as essential for promotion of 
assistant professors in Land Grant and similar universities, regardless of extent of 
outreach responsibilities. The vita, transmittal letter, and sample publications 
cover a department’s basic case for promotion, with external review letters 
assessing scholarly productivity and promise, and evaluation by students and peers 
addressing quality of teaching. All of these can contain evidence of impact on the 
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field, which was mentioned more frequently regarding outreach performance than 
for general performance. 

i 

Table 6 lists two aspects that received more mentions regarding 
performance generally than for outreach performance. 

Table 6 



Aspects That Were More Influential 
Generally Than for Outreach 

Aspect Frequency of Mention 





General 


Outreach 


* Outreach research. Evidence 
of Contribution 


27 


19 


* External Funding 


10 


5 



These two aspects are connected because of the high priority for faculty research 
at Land Grant universities and emphasis on external handing that serves many 



purposes. 
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Table 7 lists five aspects that received more mentions regarding influence 
on outreach performance recommendations than for performance generally. 



Table 7 



Aspects That Were More Influential Regarding 
Outreach Than For Performance Generally 

Frequency of Mention 



Aspect 


General 


Outreach 


* Outreach research evidence of contribution 


25 


40 


* Outreach teaching participant ratings 


27 


36 


* Outreach service 


14 


23 


* Expectations/appointment letter/position 


7 


20 


description 






* Impact and contribution to the field 


12 


17 



Understandably, outreach research, teaching, and service were among the 
most frequently mentioned aspects, especially regarding evaluation of outreach 
performance. Evidence of impact of faculty performance on the field was deemed 
influencial on promotion recommendations, as were expectations reflected in 
appointment letters and position descriptions. (For some packets, there was no 
appointment letter or position description, and expectations were very general.) 




24 



24 



As a way of exploring the likely influence of aspects on the 
recommendation, frequency of mention of the 20 aspects of packets was compared 
with levels of recommendation (deny, marginal, recommend, strong). The average 
rating for three packets was approximately strongly recommend. The average 
rating for two packets was between deny and marginal recommendation. The 
average ratings for the remaining 13 packets were grouped around recommend 
promotion. Table 8 presents the frequency distributions for eight aspects that were 
highly associated with level of recommendation, which reveal differences between 
influences on general recommendation and on outreach performance. The 
distribution for most of the remaining aspects were similar to each other. 

Table 8 

Relationships Among Aspects Most Frequently Mentioned 
As Influencing Recommendation, and Level of Recommendation 



General Outreach 



Aspect 


Hi 


Md 


Lo 


Total 


Hi 


Md 


Lo 


Total 


*External review 


8 


18 


4 


30 


6 


18 


3 


27 


*Outreach research 


3 


21 


1 


25 


7 


29 


4 


40 


*Publication 


3 


12 


4 


19 


6 


13 


1 


20 


*Discipline research 


3 


20 


4 


27 


3 


14 


2 


19 


*Resident teaching 


3 


15 


4 


22 


4 


12 


3 


19 


Transmittal letter 


6 


12 


3 


21 


3 


10 


3 


16 


*Outreach teaching 


3 


21 


2 


27 


4 


28 


4 


36 


*Expectation/appointment 


2 


5 


- 


7 


3 


15 


2 


20 




25 



25 



* Expectation/appointment 


2 


5 


- 


7 


3 


15 


2 


20 



Table 8 lists aspects that were identified previously as most frequently 
mentioned as influential on promotion decisions, regardless of the extent of an 
assistant professor’s emphasis on outreach. Included were outreach teaching and 
research, external review letters, publications, expectations regarding appointment, 
disciplinary research, and resident teaching. Aspects most frequently mentioned 
regarding performance generally included external review (30), disciplinary 
research (27), and outreach research (25). Aspects mentioned most frequently 
regarding outreach performance especially included outreach research (40), and 
teaching (36), and external review (27). Within each of these ratings of general 
and outreach performance, the high, middle and low columns of the table refer to 
the average level of the promotion recommendation (strong, recommend, 
marginal). The aspects that were especially associated with the level of 
recommendation regarding performance generally were the department chair’s 
transmittal letter (6) and the external review letters (8). The apects that were 
especially associated with the level of recommendation regarding outreach 
performance were outreach research (7), publications (6), and external review 
letters (6). 
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Table 9 

Rationale for Assessment of Packet 
^Frequency of Mention) 



Content of comments 


Q 21b 
reason for 
promotion 
recommen 
dation 


Q22b 
why 
selected 
influences 
of aspects 


Q23 

missing 

information 


Q24b 

why 

aspects for 
outreach 


Q25 

other 

comments 


1 . Quality, balance, productive, 


33 


17 


16 


10 


11 


leader 












2. Expectations, appointment, 


19 


16 


16 


14 


13 


dept. 












3. Packet, details, organization 


11 


12 


13 


13 


18 


4. Research, importance, 


25 


17 


17 


7 


5 


publications 












5. Teaching, reactions, observation 


17 


16 


15 


1 1 


3 


6. Outreach impact, evaluation, 


28 


18 


18 


29 


14 


responsive 












7. External, letters, recognition 


13 


18 


17 


17 


4 


8. Funding 


5 


3 


2 


2 


1 


9. Own assessment, future 


1 


3 


3 


3 


6 


directions 













In addition to reviewers’ mention of some of the 20 aspects of the packets 
that were listed in the review form, they commented in response to five open- 
ended questions (and in comments throughout) on their rationale for why they 
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made the assessment decisions that they did. Table 9 lists the content of their 
comments based on an inductive classification that resulted in nine categories of 
content used to code their responses to the five open-ended questions regarding: 
their reasons for their promotion recommendation, why they selected the aspects 
that influenced their recommendation, the types of information that was missing 
from the packet which would have contributed to their decision, the aspects that 
were especially relevant to assessment of outreach performance, and other 
comments. 

The most frequently written comments regarding reviewers’ reasons for 
their promotion recommendation pertained to: 

* Balanced and high quality productivity 

* Evidence of outreach impact 

* Important research and publications 

In reviewers’ comments about why they selected some aspects of 
promotion packets as especially important for recommendations related to 
outreach performance, evaluative evidence of outreach impact was prominent. 
Such evidence of outreach impact has often been suggested over the years. In 
response to this suggestion it has been noted that convincing evidence of impact is 
difficult to obtain because of multiple influences, and that comparable evidence of 
impact of resident instruction and disciplinary research is seldom provided. 

The comments about why reviewers selected the aspects they did as 
influential on the promotion recommendation generally were quite evenly 
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distributed across two-thirds of the categories of rationale (16-18 mentions). This 
was also so for comments on missing information. 

Institutional and Departmental Variations 
The foregoing sections of this report have emphasized influences on 
promotion recommendations at upper midwest Land Grant universities. This 
section focuses on variations among universities and departments. Some striking 
contrasts emerged for three aspects — appointment letters and position 
descriptions, transmittal letters for the promotion packet, and external review 
letters. The following example illustrates such contrasts. 

Dr. Stier was an assistant professor of animal science whose outreach 
activities were in several developing countries. Following B.Sc., M.SC and Ph D. 
degrees and two post-doctoral research associate positions in animal science, 

Dr. Stier served for five years as principal scientist in a full time technical adviser 
role at an international livestock center in a developing country. He continued in 
this role overseas during his first two years as an assistant professor. During the 
subsequent four years on the tenure track his position was 25% international 
outreach, 50% research, and 25% resident instruction. 

International outreach teaching and research was central throughout the six 
years, as reflected in his appointment, position description, and annual work plans. 
Included were: working abroad; technical assistance; encouraging cooperation 
among research, extension, and other organizations; encouraging on farm research; 
collaborative research; advising students working abroad, and seeking external 
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funding for such research. The departmental vote was unanimous in favor of 
promotion. 

External review letters were very positive regarding Dr. Stier’s 
effectiveness in human resource development and capacity building for applied 
research and extension in developing countries. Collaborative proposals for 
projects in other parts of the world were also developed and funded. In addition 
to numerous external grants for his work, totaling a half-million dollars, he was the 
driving force behind many large collaborative projects. His publications included 9 
articles in 5 refereed journals, 1 1 invited conference papers, and 1 1 invited 
presentations. Student evaluations were positive (average 4 on a 5-point scale) in 
his course on Livestock Production in Agricultural Development. A major 
outreach teaching activity was a 3-week summer institute on agricultural research 
to which he contributed importantly for six years. Five graduate students were 
part of his laboratory. The department chair concluded his transmittal letter by 
stating that Dr. Stier brought a new dimension to the department, that his 
combination of human resource development and sound service helped solve 
interdisciplinary problems in rural settings in developing countries, and that he fully 
met the expectations of the position for which he was hired. 

Five review forms were completed based on Dr. Stier’s promotion packet, 
and each reviewer from a different university assumed that Stier was from their 
university when assessing the packet. Four of the reviewers recommended 
promotion, and there were many similarities in the ratings of importance and 
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quality of the 20 aspects, and in their written comments. Common themes were 
that he met or exceeded expectations, and that international teaching and research 
is difficult to assess. One reviewer commented that the candidate had a very good 
outreach effort, and because the reviewer had traveled to developing countries, he 
appreciated the candidate’s accomplishments, and that colleagues without this 
experience might view the packet differently. Contrasting perspectives occurred 
regarding position description, transmittal letter, and external review letters. 

Most reviewers commented that their review was helped by an 
understanding of Dr. Stier’s appointment letter, position description, and 
expectations for his performance, as they judged his performance, especially 
because his international outreach was unusual. However, one reviewer 
commented that such information was important mainly at the time of hiring. The 
reviewer who did not recommend promotion commented that such a decision 
should not be based on the written materials in the packet alone, but should reflect 
personal interaction. 

Most reviewers considered the transmittal letter from the department chair 
as essential to indicate extent of departmental support and the context in which the 
candidate was working. By contrast, several reviewers indicated that they do not 
use such a transmittal letter to convey the departmental assessment. 

The use of external review letters based on a review of the promotion 
packet and sample publications was widespread and most of these five reviewers 
rated them as important to include and those for Dr. Stier reflected his excellent 
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performance, especially so because so much of his performance was away from 
campus. By contrast, one reviewer indicated that such external review letters are 
not used at his institution (although packets and reviewers from other departments 
at that university do so). Another reviewer questioned the utility of external 
reviews generally because such reviewers are dependent on the written materials 
and lack first-hand observation and interaction with the candidate. 

Another college of agriculture candidate also illustrates such contrasting 
perspectives that reflect institutional differences more than reviewer conclusions. 
Dr. Utter was an assistant professor of dairy science whose outreach (extension) 
activities were within the state, and whose average level of promotion 
recommendation was the highest in the study. 

His appointment during the six years was 75% as an extension specialist, 
and 25% research, with no resident instruction appointment. However, he was 
very effective mentoring graduate students, with guest lecturers in courses, and he 
helped to develop and teach in a capstone course for undergraduate majors. 

The departmental executive committee was unanimous in support of 
promotion. His extension work was highly regarded by extension staff and various 
practitioners related to the dairy industry. His outreach teaching was outstanding 
and much appreciated. He initiated, coordinated, and taught in various 
conferences and seminars for practitioners; in-service sessions for county extension 
staff, and during his probationary period made almost 50 presentations to 
international, national, and state extension conferences; in addition to more than 
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200 state and county meetings. He also produced single-print publications (50), 
popular press articles (60), computer software programs (10), videotapes (5), and 
video sattelite programs (3). His applied research was supported by 20 grants 
totaling about a half million dollars and resulted in 7 research articles in refereed 
journals. Several external reviewers referred to Dr. Utter as one of the top 
extension specialists in his field. One reviewer rated almost every aspect as of the 
highest quality and in commenting on the strong recommendation for promotion 
referred to the clear, consistent, and strong record. The balance of teaching, 
research and service was rated high, “as defined by the position description.” 
However, this reviewer wrote that a transmittal letter and external review letters 
were not used at that institution. (By contrast, a reviewer in another college at the 
same institution wrote that external review letters are very important for review, 
and rated the transmittal letter as very important also.) 

Another reviewer decried over-emphasis on written materials when 
deciding on promotion, and urged personal interaction and documentation to 
confirm assertions in the packet. 

By contrast, one of the four reviewers commented that the transmittal letter 
from the department chair was the single most important document in the packet. 

If a department chair does not write a very supportive letter, hidden problems are 
suspected. Information from external reviewers and the clientele can validate 
assertions in the packet. This reviewer characterized this candidate as almost too 
good to be true, without real weaknesses. 
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Other reviewers of other candidates who commented on external reviewer 
letters also expressed contrasting viewpoints. One stated that the validity of an 
external review depends on a sound understanding of the candidate’s position. 
Another reviewer generally discounts external reviews because they tend to be 
hand-picked and likely to be positively disposed. 

The foregoing examples illustrate some of the contrasting viewpoints that 
reviewers have, which reflect institutional and department practices and 
expectations, in addition to variations among individual reviewers based on their 
own experience and values. This variation suggests that candidates and others 
involved in the promotion and tenure process should treat general guidelines as 
tentative. This is especially so because in most institutions major guidelines and 
expectations are often implicit and not in writing. This applies even to variations 
among departments in a university. If considered tentative, guidelines can guide 
exploration to assess the expectations and past decisions related to a department 
and other people beyond the department who contribute to the review process. 
Consistent expectations reported by people familiar with institutional procedures 
can help validate actual expectations. 

Discussion 

This study report culminates more than a quarter century of the author’s 
interest in encouraging and recognizing excellent faculty performance in university 
outreach forms of teaching, research, and service. During this time there has been 
a dramatic increase in interest in evaluating faculty outreach performance. This 
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trend reflects growing recognition of the extent and centrality of university 
outreach, extension, continuing education and public service. 

A major influence on institutional attention to faculty outreach roles was 
from Boyer’s Scholarship Reconsidered (1990) which was further refined by Rice 
and Richlin (1993). Scholarship Assessed (Glassick, Huber, and Maeroff, 1997) 
suggests criteria for evaluation that pertain to assistant professors with outreach 
responsibilities. Increased attention to the scholarship of teaching and application 
of knowledge (in addition to discovery and integration) enhanced the rationale for 
outreach. Previous rationales for outreach teaching and research contributed, such 
as Enarson (1989) and Lynton and Elman (1987), which Lynton (1995) has 
continued to refine as professional service. Many subsequent publications reflect 
increasing centrality of outreach as a professional role, such as Diamond and Adam 
(1993), Curry and Wergin (1993), Wergin (1994) and JDSO (1996). 

The process of assessment and rewards has been a continuing challenge 
over the years. What criteria should be used to judge the quality of faculty 
outreach performance, what procedures should be used to obtain information on 
which to base evaluative judgments, and how should evaluative conclusions be 
reflected in recognition and rewards such as promotion and tenure? Examples of 
publications that explore such rationales and procedures include Centra (1979, 
1993) and Braskamp and Ory (1994). 

Evaluation of the quality of outreach performance reflects more than the 
evidence presented. Such judgments regarding merit increases and promotion also 
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reflect institutional and colleagial values, beliefs, and expectations that are seldom 
explicit. Promotion guidelines are insufficient when they do not fit the implicit 
priorities of people actually making decisions. Examples of writings that pertain to 
the influence of collegiate culture on evaluation of faculty outreach performance 
include Florestano and Hambrick (1984), Eastman (1989), and Austin (1992). 

One important part of the assessment process is the decision process of 
reviewers. In Land Grant and other large research-oriented universities, 
promotion materials are reviewed at department, college, and university levels. 
There have been some studies that have analyzed the criteria and recommendations 
of reviewers of fictionalized but authentic promotion materials. (Salthouse, 
McKeachie, Lin, 1978; Cohen and McKeachie, 1980; Lin, McKeachie and Tucker, 
1984). The relative weight that reviewers gave to faculty research, teaching and 
service roles was the main influence on promotion recommendations and not the 
type of information provided about roles that were not highly valued (Salthouse, 
McKeachie, Lin, 1978). 

The groundwork for this study was laid in the early 1970s at the University 
of Illinois at Urbana-Champaign. Senate committee recommendations regarding 
recognizing excellent faculty performance in continuing education and public 
service were unanimously approved. Implimentation of the recommendations 
occurred as part of a reorganization of the outreach function to become more 
decentralized and reflective of faculty concerns (Knox, 1975). Criteria for 
evaluating excellence of faculty outreach performance were developed and 
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introduced, along with efforts to encourage their actual use by reviewers (Votruba, 
1978; Hanna, 1981). An update has been shared with colleagues at similar 
universities with much benefit (Illinois, 1993). However, in the mid-1970s, efforts 
to explore this issue with colleagues at other institutions met with general 
disinterest. By contrast, in the mid-1990s, there was great interest in various 
higher education associations regarding assessing and rewarding faculty 
performance in various roles including outreach. 

During the 1990s, Land Grant universities were especially active in 
exploring rationales and procedures for encouraging and recognizing excellent 
faculty outreach performance. Especially prominent in this effort were presidents 
and provosts, and members of councils and committees concerned with outreach 
(extension, continuing education, public service), which often included 
administrators with outreach responsibilities. At the University of Wisconsin- 
Madison, the Council on Outreach developed a rationale, which contributed to 
revision of campus-level guidelines for promotion and tenure to produce more 
positive guidance regarding outreach and extension (Wisconsin, 1997). A related 
ad hoc committee encouraged initiation of this study. Similar developments were 
occurring at universities in other states, such as: Georgia (1989), California (Pister, 
1991), Kansas (1992), Illinois (1993), Michigan (1993), Minnesota (1993), 

Oregon (1994), and Pennsylvania (1994). Through participation in collaborative 
projects, national higher education association meetings, and invitational 
conferences, there has been much cooperation and exchange that has benefited this 
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study and efforts generally to strengthen assessment of faculty outreach 
performance. 

The purpose of this study was to better understand the process of 
reviewing and judging promotion packets for assistant professors with varying 
levels of outreach responsibilities. Information from the fictionalized packets and 
from the review form completed by reviewers was analyzed in various ways to 
identify aspects of the packet that were associated with a positive 
recommendation. Following are the main themes that emerged across the several 
analyses. 

The cluster analysis resulted in three clusters of variables. One cluster, 
which included the promotion recommendation, included two sub-groups. One 
sub-group, labeled creatively responsive, was composed of the promotion 
recommendation and quality aspects of the packet that pertained to responsiveness 
to university outreach/extension expectations and to originality, creativity and 
innovation. No other analyses identified these two aspects as related to the 
promotion recommendation. However, the second sub-group, labeled impact on 
the field, was composed of the quality aspect on impact along with external review 
letters and importance of the topics that candidates addressed. Both impact and 
external review letters (which typically address and document impact) emerged 
from reviewers’ listings of aspects that influenced their promotion recommendation 
and from their rationales for their judgment. Topic importance could have 
contributed to both impact and external reviewers’ assessments. 
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The second major cluster was related to career balance of research as well 
as teaching and service. Six of the ten quality aspects in this cluster also emerged 
from other analyses which indicated their influence on promotion 
recommendations. Outreach research and sample publications that reported such 
findings were included in all other analyses, so deserve special attention. In 
addition to the apsect on balance and the one on the vita (which contained 
documentation on teaching, research and service), quality aspects in this cluster 
included discipline oriented research, outreach service, and career improvement 
and enhancement. It appeared that reviewers valued promotion packets that 
reflected such career balance of research as well as teaching and service. At most 
research-oriented universities, outreach research deserves special attention by 
assistant professors with outreach responsibilities. 

The third major cluster reflected role expectations, especially related to 
teaching. Three quality aspects in this cluster also emerged from all other analyses. 
They were: expectations as indicated in the appointment letter and position 
description, resident instruction and outreach teaching. The remaining two quality 
aspects emerged from two of the other analyses. They were the transmittal letter 
from the department chair (which reflected departmental expectations), and 
external funding (which typically reflected college expectations). 

In general, the triangulation from the analyses provides some cross 
validation regarding the quality indicators valued by reviewers of promotion 
packets of assistant professors with varying levels of outreach responsibilities. In 
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the cluster analysis, the promotion recommendation was part of the sub-groups of 
quality aspects that reflected evidence of being responsive to university outreach 
and having an impact on the field. The collaborative and long-term nature of many 
outreach activities creates a special challenge to evaluation and documentation of 
such impact, but makes it important to include some assessment. Such impact 
evaluation can constitute outreach research. The two other major clusters that 
were separate from the promotion recommendation, were career balance that 
includes outreach research and role expectations that include outreach teaching. 
These two clusters were separate from each other, but the quality aspects in each 
emerged from the other analyses as influential on reviewers’ recommendations. 
This suggests the desirability of including information related to quality of 
performance regarding each of these two clusters in promotion packets, in addition 
to evidence of impact and responsiveness. 

Level of promotion recommendation (deny, marginal, recommend, strong) 
was little related to the extent of outreach as part of the total appointment. 
However, the aspects that were valued by reviewers were more likely to occur for 
assistant professors whose appointment included research and resident instruction. 
However, some of the candidates’ packets that were most highly rated by 
reviewers reflected much of the effort devoted to outreach forms of teaching and 
research. It is likely that most of the packets would have been recommended for 
promotion to associate professor with tenure, even if outreach and extension was 
ignored. However, in many but not all instances reviewers accepted outreach 
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forms of teaching, research and service as legitimate, instead of traditional reliance 
on resident instruction, discipline-oriented research, and institutional service. The 
quality of performance and documentation that included teaching, research and 
service seemed to be more important than relative emphasis on outreach. 

Fortunately, the criteria that emerged from this study were very congruent 
with the six general standards proposed by Glassick, Huber and Maeroff (1997, 
p.36) for evaluation of the professoriate. Their six standards pertained to clear 
goals, adequate preparation, appropriate methods, significant results, effective 
presentation, and reflective critique. The Creatively Responsive cluster from the 
current study reflected clear goals (importance of topics, innovative, responsive to 
expectations which should be apparent from the transmittal letter from the 
department chair); adequate preparation (evidence from vita and other sources 
regarding career improvement); and appropriate methods (methodological rigor in 
outreach and disciplinary research). The Impact on Field cluster reflected 
significant results (impact, outreach teaching, honors). The Balance of Research 
as well as teaching and service cluster reflected effective presentation 
(publications, external review letters). The Expectations cluster reflected reflective 
critique (career improvement as indicated in the vita). 

Beyond these general conclusions, it is important to recognize the 
contrasting viewpoints among some reviewers regarding the applicability of 
aspects such as appointment/position description, transmittal letter from 
department, and external review letters. This suggests the desirability of using 
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promotion guidelines as tentative until they are compared with actual expectations 
and past decisions in the specific situation. 

In conclusion, the study findings suggest the following implications for 
strengthening the process of assessing and recognizing excellent university faculty 
outreach performance. 

1. Throughout the probationary period, assistant professors and those 
who mentor and assist them should emphasize, evaluate, and 
document those activities and accomplishments that they and 
reviewers are likely to value at the time of a decision on promotion 
and tenure. 

2. These guidelines can be useful to people in various roles related to 
a promotion decision, including: candidates, mentors, department 
chairs, reviewers and outreach administrators. 

3. Include at the successive stages of the evolving promotion packet, 
information about aspects of high quality performance likely to be 
especially relevant to the promotion decision, such as the following: 

A. Creatively responsive to university outreach (responsive, 
creative). 

B. Impact on the field (impact, external review). 

C. Balance of research with teaching and service (balance, 
outreach research and service, sample publications, 
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disciplinary research). It may be desirable to explain the 
collaborative nature of some outreach research. 

D. Expectations, especially teaching (letter of appointment, 
transmittal from department, outreach teaching). 

4. Make the promotion packet as brief, complete and clear as possible 
to avoid unnecessary material and emphasize information most 
pertinent to the promotion decision regarding quality and 
productivity. 

5. There has been gradual acceptance of outreach forms of teaching, 
research, and service as legitimate with both similarities and 
differences compared with disciplinary research, resident instruction 
and institutional service. 

6. The proportion of outreach within an assistant professor’s total 
contribution is less important for promotion than convincing 
evidence of balanced attention to high quality performance and 
accomplishments. 

7. Societal and institutional trends and differing experiences and 
expectations can result in contrasting reviewer viewpoints so treat 
any general guidelines as tentative and assess actual current 
expectations and past decisions. 

8. When deciding on information to emphasize in a promotion packet, 
consider likely institutional expectations about the importance of 
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various aspects of the packet, as well as evidence of the quality and 
impact of performance. 

9. Universities can revise promotion guidelines, but the values and 
expectations of reviewers may not match the guidelines. 

10. Recognize and use the increasing amount of material relevant to the 
promotion and tenure process to provide background to guide 
career advancement for assistant professors with outreach 
responsibilities. 
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Review Form 1/97 



ASSESSMENT OF PROMOTION PACKET 



Packet Number Fictional Name Field 

Thank you for helping us understand criteria and procedures related to the promotion and 
tenure decision for assistant professors with outreach responsibilities. Please complete a separate 
review form for each fictionalized promotion packet, as you might if you were part of an actual 
promotion review process at departmental, college or university levels. 

Part A of the review form is on specific aspects of the promotion packet, Part B is on your 
overall impressions, and Part C is on your recommendations. 

Part A 



For each aspect listed in Part A, please complete two ratings: (a) Importance (how 
important is such information generally for a promotion decision?) and (b) Quality (how high is 
the quality, as reflected in the information provided in this packet?) Complete each rating by 
circling the number that best represents your judgment. (If there is inadequate information to 
make a judgment, circle NA.) Provide explanatory comments. 



1 . 



2 . 



3. 



4. 



5. 



ASPECT 

Transmittal letter from department chair. 
Comments: 



Expectations/appointment letter/position description. 
Comments: 



Vita/background information on education, career. 
Comments: 



Resident teaching/peer revie w/student ratings. 
Comments: 



Outreach teaching participant ratings. 
Comments: 





Lo 








Hi 




(a) Importance 


1 


2 


3 


4 


5 


NA 


(b) Quality 


1 


2 


3 


4 


5 


NA 


(a) Importance 


1 


2 


3 


4 


5 


NA 


(b) Quality 


1 


2 


3 


4 


5 


NA 


(a) Importance 


1 


2 


3 


4 


5 


NA 


(b) Quality 


1 


2 


3 


4 


5 


NA 


(a) Importance 


1 


2 


3 


4 


5 


NA 


(b) Quality 


1 


2 


3 


4 


5 


NA 


(a) Importance 


1 


2 


3 


4 


5 


NA 


(b) Quality 


1 


2 


3 


4 


5 


NA 
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ASPECT Lo Hi 



6. Discipline oriented research, pattern of publications, (a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


7. Outreach research, evidence of contribution. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


8. Institutional service, committees. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


9. Outreach service. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


10. External funding. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


1 1 . Sample publications. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


12. External review letters. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 



Part B 

The following ratings in Part B seek your overall impressions based on information from 
various aspects of the promotion packet. 

ASPECTS Lo Hi 

13. Responsive to university outreach/extension 

expectations. (a) Importance 1 2 3 4 5 NA 

Comments: 

(b) Quality 1 2 3 4 5 NA 
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14. Balance of teaching, research, service. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


15. Importance of topics that 


candidate addressed. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


16. Originality, creativity, innovation. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments. 




(b) Quality 


1 


2 


3 


4 


5 


NA 


17. Methodological rigor. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


18. Impact and contribution to field. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


19. Honors and recognition. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 


20. Career improvement and enhancement. 


(a) Importance 


1 


2 


3 


4 


5 


NA 


Comments: 




(b) Quality 


1 


2 


3 


4 


5 


NA 



PartC 



Your recommendation and comments. 

21. What decision would you recommend regarding promotion to associate professor with tenure 
for the nominee, and why? 

(a) promote deny 

(b) why? 



22. Which aspects of this candidate’s packet most influenced your recommendation, 
and why? 

(a) (aspect numbers) 

(b) why? 
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23. What missing information (if any) should have been included, to contribute to a sound 
judgment? 



24. In general, which aspects of a promotion packet are most important to you in judging the 
performance of a faculty member with outreach responsibilities, and why? 

(a) (aspect numbers) 

(b) why? 



25. Other comments: 



Reviewer Name Institution 

Address 



Return to: Alan B. Knox 

University of Wisconsin 
276 TEB, 225 N. Mills St. 
Madison, WI 53706 




52 



wcin i ay: 



; l-lb-yy Ill-UWM fUW-MADISON-tD, ADMIN, -+ 



JAM. 14,1999 etSBAM ERIC HE 




eric facility;# 2 



NO. 252 Pi 2^3 



MDvtitmdNuction 



“S?ST EHIC 



r 



reproduction release 

(Bpadflc Document) 

DOCUMENT tgENggjgATm 



tw#i mania* hcsuent outreach kwowcanckT 



Author^: >. nip* 

corpora# Source: 



PubllceBon Data; 

D^oeafeau 1095 




IL REPRODUCTION RELEASE! 



on. or m, mimimimi '£2*^SZ££ D "*>- i» eH*> * 

at mapIST* 1 ** *""" “ rBprDdl1 " w8 «»*»*«» v wnww amrm, mm CHsex owe 



)optlef|« nttiSf) It 0# Bottom 



TO JWftfiOUCt AND' i > m 

pmnwMTB mV matcwm. h * 
aaNftfcsNTMiv 



y* 



"mssawr 



Umi 

r 



W .HRSEMP2S"«* 

ateftaMai&iMiwrti 



MBRBgaar~ 

z~ ^ 



-nimmM 




Isa 






□ 



UvolM 

T 



~^»SWiWjsBBsa.«. 




1* 



n&aovM&Q 



u vsltl 



wmnui 

s rw " 



“SSSESSS 28 



□ 

”t5SSS~! 



“^^■i^Egi^asaatfs 



□ 

■JS®«aBsnas! 



raAMEULMMi 



^^SSSSy^ ss; »» (kra j 

to^/WbrmMM wmi or 









ZWP'sea-aM? 


Hs8#-2Ss-in5 ' 



ERIC 



0WM 



